s3: Filter S3 profiles to only those used by the application - #503

Keep the previous code - the only difference is the profiles used to gather. The old code gathered from all profiles, we need to gather only from the application profiles for each application.

The simplest way is to do the gather per failed application instead of single gather for all applications:

for app in failed appes:
    get the prefix for tha app
    get the profiles for the app
    gather s3 data for profiles and prefix

But since we use the same drpolicy for all test apps - we will have the same profiles names for every app - so we can do:

 get prefixes for all apps
 get profiles for all apps (removing duplicates)
 gather s3 data from profiles and prefixes

This is a good start - change the function we call to get the application profiles instead of all profiles!

But we pass 2 many arguments like configMapName and configMapNamespace - this values comes from ramen package, so ramen already know them, so no need to pass them.

Need rethink passing these arguments:

• c - the command - is it for the logger? we should pass the logger instead.
• reader
• c.outputReader - would be better to implement this in the ramen package and not pass functions like this
• configMapName, configMapNamespace - ramen already know these.

@nirs

1. ctx is used not for logging. It's needed currently for PrimaryCluster(ctx, drpc) which calls ctx.Env().GetCluster()
2. c.outputReader: not sure how, add OutputReader(string) to the ramen.Context interface?
3. configmap:
   • configmap Name: Yes, but existing configMapName := ramen.HubOperatorConfigMapName is passed from caller, refactor this code to directly use?
   • Configmap ns: this comes from user config config.Namespaces.RamenHubNamespace varies between k8s and ocp

ctx is good - matching how we pass commands everywhere - keep it.

The hub reader is already used in other functions, but current coded needed single cluster, and here we need to read form 2 clusters. Reading clusters requires the command dataDir(), so we added outputDir() since it was used only in the command command. Passing it to the ramen code works but not great.

If we add OutputReader(string) to the context interface, ramen code can create readers from the context so we don't need to pass the hub and the outputReader function. This seems useful since ramen package need to read stuff from the output directory.

The ramen package knows the ramen hub config map name so we don't need to pass it.

The context has the config, so the ramen package can find the config namespace.

With these changes the call will become:

storeProfiles, err := ramen.ApplicationProfiles(c, c.opts.DRPCName, c.opts.DRPCNamespace)

Lets try do this:

1. commit 1: Add OuptutReader to ramen context
2. commit 2: Add ApplicationS3Profiles (this commit)

If adding OutputReader() is too big we can pass the function and let ramen.ApplicationS3Profiles create the readers.

We can change ramen.ApplicationS3Prefix() to accept a context later for creating the reader instead of passing the reader. This can be a followup PR in main. release 0.23 will have less consistent code.